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INTRODUCTION 


Elementary teachers face formidable obstacles when planning and implementing science 
instruction, including inadequate preparation opportunities, lack of resources, and accountability 
pressures. Data from the 2012 National Survey of Science and Mathematics Education bear this 
out (Banilower et al., 2013). Further, the expectations for elementary science instruction were 
raised to a new level with the release of the Next Generation Science Standards (NGSS Lead 
States, 2013),the latest in a series of college and career-ready standards. Together with the 
Common Core State Standards in Reading and Mathematics (National Governors Association 
Center for Best Practices & Council of Chief State School Officers, 2010a, 2010b), they put forth 
an ambitious vision of what students should know and be able to do in these fields as a result of 
K-12 education. To realize the vision of excellent science education for all students portrayed in 
the NGSS, elementary teachers will need to draw on varied domains of knowledge. Prominent 
educators and researchers have proposed the existence of a professional knowledge base for 
teaching, similar to the specialized knowledge bases for medicine and law (Grossman, 1990; 
Hiebert, Gallimore, & Stigler, 2002; Hill, Rowan, & Ball, 2005; Shulman, 1986). Efforts to 
articulate the components of such a knowledge base have been underway for over two decades. 


Some constituent knowledge forms, such as disciplinary content knowledge, are fairly well 
understood and widely accepted as necessary, but not sufficient, for effective teaching (e.g., 
Heller, Daehler, Wong, Shinohara, & Miratrix, 2012). Perhaps the most widely recognized form 
of specialized knowledge for teaching—and arguably the one with the most potential for helping 
teachers overcome knowledge-related obstacles—is “pedagogical content knowledge” (PCK), 
which Shulman (1986) described as an amalgam of pedagogical knowledge (general teaching 
knowledge) and content knowledge (knowledge of a specific discipline). An oft-cited example is 
knowledge of an effective strategy for teaching a particular concept; for example, having 
students slide an object on progressively smoother surfaces to construct an understanding of the 
idea that an object in motion tends to remain in motion in a straight line unless a force acts on it. 
Magnusson, Krajcik, and Borko (1999) developed a model of PCK that has strongly influenced 
conceptualizations of what constitutes PCK in science as well as other disciplines. Recently, a 
new model of PCK emerged, one that acknowledges both collective and personal aspects of PCK 
(Gess-Newsome, 2015). In this model, shared or collective PCK is referred to as topic-specific 
professional knowledge (TSPK). Hypothesized relationships among these and other forms of 
knowledge are shown in Figure 1. 


As illustrated in the model, discrete professional knowledge bases—disciplinary content 
knowledge chief among them—are the foundation for TSPK. Examples of TSPK include an 
instructional strategy that has been found through empirical studies to be effective for teaching a 
specific idea, or recognition of a conceptual difficulty found through assessment studies to be 
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prominent among elementary students. This knowledge can be applied by teachers to their own 
unique settings and for their own purposes. As teachers take up TSPK—e.g., through reading, 
professional development experiences, discussions with colleagues, reflecting on their practice— 
and use it in their teaching, it becomes personal PCK. 


PCK Model 


Teacher Professional Knowledge Bases 


Assessment Pedagogical Content Knowledge of Curricular < 
Knowledge Knowledge Knowledge Students Knowledge 


Topic-Specific Professional Knowledge (TSPK) 
>| Knowledge of: instructional strategies, content representations, 
student understandings, science practices, and habits of mind 


Amplifiers and Filters: teacher beliefs, 
orientations, prior knowledge, and context 


Classroom Practice 


«i» Classroom Context 
EEA S H (Curriculum, etc.) 


Figure I 


TSPK can help elementary teachers overcome knowledge-related obstacles to science teaching in 
several ways. Most importantly, TSPK provides a rich resource for helping teachers incorporate 
what is known about effective teaching of a topic into their instruction (see Figure 2). For 
example, TSPK can be a valuable instructional planning resource or the focus of discussion in a 
teacher study group or professional learning community. Another high-leverage use of TSPK is 
in instructional materials development (Banilower, Nelson, Trygstad, Smith, & Smith, 2013). 
Similarly, teacher educators and professional development providers can use TSPK to craft and 
provide topic-specific support for pre-service and in-service teachers. 
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TSPK Theory of Action 


Instructional 
Materials 


Teacher 
Planning and 
Instruction 


Student 
Learning 


Teacher Professional 
Development 


Pre-service Teacher 
Preparation 


Figure 2 


There is a common perception that TSPK is widely available. Although TSPK does exist and 
has been compiled in a few topic areas (e.g., empirical research abounds for student thinking 
about force and motion), most science topics are not yet well researched. Even a brief search of 
the literature illustrates the lack of easily accessible TSPK in many topics. In addition, the 
literature that does exist is not organized for use by teachers. 


With support from the National Science Foundation, Horizon Research, Inc. (HRI) is testing a 
method for collecting and synthesizing PCK from multiple sources, with the ultimate goal of 
making the resulting TSPK available to teachers as a support for implementing the NGSS. The 
method draws on empirical research literature, practice-based literature (e.g., professional 
journals for classroom teachers), and expert wisdom of practice (collected by surveying and/or 
interviewing expert practitioners). 


In this report, we describe the results of a review of empirical research literature related to 
teaching one topic from the NGSS. Our goal was to determine how much PCK for teaching 
interdependent relationships in ecosystems at the upper elementary level could be “extracted” 
from the empirical literature. Subsequent reports will describe efforts to synthesize PCK from 
the combination of this source, the practice-based literature, and expert wisdom of practice. 


METHODOLOGY 


The literature review focused on one NGSS disciplinary core idea (DCI) related to 
Interdependent Relationships in Ecosystems at the fifth grade: 5-LS2.A. The NGSS state the 
ideas related to interdependence as: 
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The food of almost any kind of animal can be traced back to plants. Organisms are 
related in food webs in which some animals eat plants for food and other animals eat the 
animals that eat plants. Some organisms, such as fungi and bacteria, break down dead 
organisms (both plants or plants parts and animals) and therefore operate as 
“decomposers.” Decomposition eventually restores (recycles) some materials back to the 
soil. Organisms can survive only in environments in which their particular needs are met. 
A healthy ecosystem is one in which multiple species of different types are each able to 
meet their needs in a relatively stable web of life. Newly introduced species can damage 
the balance of an ecosystem (NGSS Lead States, 2013, p. 48). 


We rearranged and unpacked some of the ideas in a way that would more easily allow us to 
organize findings from the literature: 


1. The food of almost any kind of organism can be traced back to producers such as 
plants and algae. 
a. Food provides organisms the materials and energy they need to grow and 
function. 
b. Producers make their own food inside themselves using energy from the sun, and 
matter from air and water. 


2. Organisms in ecosystems are related in food webs 
a. Consumers get their food by eating other organisms. Some consumers eat 
producers. Some consumers eat other consumers. 
b. Decomposers, such as bacteria, fungi and earthworms, are consumers that break 
down dead organisms (or parts of organisms). 
c. Decomposition eventually restores (recycles) some materials back to the 
environment, making necessary resources available to producers. 


3. Organisms can survive only in environments in which their particular needs are 
met. Environmental conditions include, but are not limited to, light, temperature, 
moisture, amount of oxygen, nutrient availability, and salinity. 


4. A healthy ecosystem is one in which the needs of multiple types of organisms are met 
in a relatively stable web of life. 


5. Natural events and human activity can change the balance or stability of an 
ecosystem. When the balance, or stability, of an ecosystem changes, the 
opportunities for different types of organisms to meet their needs can increase or 
decrease. 
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We began the literature search by identifying a list of key search terms, such as “ecosystem(s),” 
“food web,” and “student knowledge.” The full list of key terms can be found in Table 1. 
Individually, these terms returned a broad spectrum of results from several search engines (ERIC 
and Google Scholar among them); therefore, many of the terms were used in combination in 
order to narrow the literature to articles that relate to teaching interdependence in ecosystems at 
the elementary school level. Two examples include, “ecosystems AND elementary students” 
and “food web AND misconceptions’ AND elementary education.” For search terms more 
likely to yield instruction guidance (e.g., lesson plans, instruction), phrases such as 
“elementary/secondary education” and “elementary/secondary science” were used to narrow the 
search. To be included in the review, a study had to meet the following criteria: 


e Reported in a peer-reviewed journal, peer-reviewed conference proceedings, or an 
edited book; 

e Included K-8 students in the study sample; 

e Was a systematic empirical study (strictly theoretical pieces were not included); and 

e Could not be a literature review only (however, the bibliographies of these articles 
were used to identify primary sources). 


Table 1 

Key Search Terms 
Activities Ecology misconceptions Elementary student Predator 
Concepts Ecosystem interdependence (ies) | Food web Prey 
Curriculum Ecosystem misconceptions Instruction Producer 
Decomposer Ecosystem(s) Learning Student knowledge 
Decomposition Elementary Lesson plans Student thinking 
Ecology Elementary education Lesson(s) Understanding 


Articles were initially screened by reading only the abstract. Those that appeared to meet the 
review criteria (N = 110) were saved in a reference management program. 


The project team created a list of tags to be applied as the articles were read more carefully, and 
the tags were used to filter the collection. For example, articles that were found to focus on high 
school students, pre-service teachers, or in-service teachers (but not K—8 students) were excluded 
from the final collection of literature. Because we were most interested in finding PCK related 


' In this report, “misconception” is used to denote any idea that conflicts with accepted scientific ideas about a 
phenomenon, acknowledging that such ideas are neither good nor bad and may represent a productive step in a 
student’s learning progression. 
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to targeted ideas about interdependent relationships in ecosystems, we also excluded articles 
focused on: 


e Specific types of ecosystems (e.g., longleaf pine, watershed, coral reef) when 
connections to general ecosystem concepts were not evident; 

e A particular set of organisms (e.g., insects, shrimp, polar bear) when connections to 
general ecosystem concepts were not evident; 

e Misconceptions or concept development at a level broader than ecosystems (e.g., 
biology, 5 grade science); or 

e Photosynthesis in the absence of its role in ecosystem. 


Although the initially screened articles spanned a wider range of dates, the final pool included 38 
studies, with publication years from 1982 to 2014 (median = 1998). 


Once the literature pool was finalized, researchers began coding the kinds of PCK in each study. 
The coding scheme was both a priori, based on the Magnusson et al. (1999) model of PCK, and 
emergent. Magnusson et al. describe discrete forms of PCK, including knowledge of 
instructional strategies, knowledge of students’ understanding of science, and knowledge of 
science curriculum. In some cases, we elaborated on these forms, for example adding 
misconceptions and learning progressions as categories of knowledge of student understanding. 


Because the articles in the pool varied in the quality of the research designs, the project team 
added a confidence rating for each piece of PCK coded. The codebook is included in the 
Appendix. This rating was intended to reflect the reliability and generalizability of the PCK. 
For example, if an article claimed that upper elementary students are likely to have a particular 
misconception, the rating conveys how confident we are that the claim is true. To determine the 
confidence rating, we used an adapted version of the Standards of Evidence (SoE) review 
process (Heck & Minner, 2009). The SoE review assesses the extent to which key components 
of the research are documented and judges support for findings considering each question that 
the publication addresses. Key in the assessment of rigor is the consideration of multiple aspects 
of internal validity. We selected a subset of indicators in order to create an abbreviated review 
process, focusing on five factors: 


1. Sample size; 

2. Appropriateness of analyses; 

3. Validity and reliability of research instruments; 
4. Appropriateness of generalizations; and 

5. Potential for investigator bias. 
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Additionally, we looked at the alignment of the purpose of the research and the coded PCK when 
determining the confidence rating. For example, if a study was not designed to identify student 
misconceptions, but student misconceptions were identified incidentally, each coded 
misconception received a low confidence rating. However, if the same item of PCK was 
extracted from many articles in the literature pool, even if it received a low confidence rating 
each time, the item ultimately received a high rating overall based on the accumulation of 
evidence. 


A codebook (included in the Appendix) was developed to provide descriptions of each code, 
rules for when a code should be applied, and explanations of how to determine confidence 
ratings. The project team iteratively reviewed 10 articles, applied the coding scheme (comparing 
codes and confidence ratings), and refined the codebook. This process was continued until the 
project team reached a consensus understanding of the codes and applied them consistently. 
Following modifications to the codebook, two project researchers applied the coding scheme to 
the remaining articles in the pool. 


Although traces of all PCK types were found in the literature, our findings reported here focus 
exclusively on student thinking and prompts because these two areas were by far the most 
prevalent across the studies. We define prompts as questions or tasks that teachers would pose to 
their students in order to elicit their thinking in writing or orally to use for formative purposes. 
Although we identify many prompts in the empirical literature, some that are appropriate for 
research purposes (e.g., one-on-one interviews) are not appropriate for classroom use. For this 
reason, we extracted prompts from the literature that the reviewer could envision a teacher using 
with students as is—that is, without substantive changes to the language used. 


In the next section, we summarize the substantive findings from the studies. 


FINDINGS 


As mentioned previously, the search and screening processes yielded 38 studies, shown in Table 
2. An expanded version of the table, including a brief description of each study, appears in the 
Appendix. 
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Table 2 


Article Summaries 


Where the Age/Grade 
Author (Year) study occurred Number of subjects 
1. Adeniyi (1985) Nigeria 232 tested; subset of | Ages 13-15 
26 students 
interviewed 
2. Alao & Guthrie (1999) US. 72 5m grade 
3. Bailey & Watson (1998) England ~100 Ages 7-11 
4. Barman, Stein, McNair, & Barman (2006) U.S. & Canada ~2400 K-83" grade 
5. Bell-Basca, Grotzer, Donis, & Shaw (2000) U.S. 60 3rd grade 
6. Brody (1993) US. 316 4", 8", and 11™ grade 
7. Brody (1994) USS. 467 4", 8", and 11™ grade 
8. Brody & Koch (1990) US. 187 4", 8", and 11™ grade 
9. Cetin (2007) England & Turkey 96 Ages 14-15 (7™ grade) 
10. Demetriou et al. (2009) Cyprus 46 Ages 9-10 (4" grade) 
11. Ero-Tolliver, Lucas, & Schauble (2013) US. 23 1“ grade 
12. Evagorou, Korfiatis, Nicolaou, & Cyprus 13 5" and 6™ grade (Ages 
Constantinou (2009) 11-12) 
13. Gallegos, Jerezano, & Flores (1994) Mexico 506 4n_en grade 
14. Gotwals & Songer (2010) US. 318 6" grade 
15. Grotzer (2009) U.S. 99 6' grade 
16. Grotzer & Basca (2003) US. 60 3rd grade 
17. Helldén (1998) Sweden ~25 2>4_gi orade 
18. Hmelo-Silver, Eberbach, & Jordan (2014) US. 311 Middle school 
19. Hogan (2000) U.S. 52; 16 focal students 6! grade 
20. Hogan (2002) US. 24 8" grade 
21. Hogan & Fisherkeller (1996) US. Curriculum piloted in | 5™ and 6" grade 
12 classrooms; 8 
students interviewed 
22. Jordan, Brooks, Hmelo-Silver, Eberbach, & U.S 66; data analyzed from 7. grade 
Sinha (2014) 35 students who 
completed all tasks 
23. Jordan, Gray, Brooks, Honwad, & Hmelo- US 40 T grade 
Silver (2013) 
24. Jordan, Gray, Demeter, Lui, & Hmelo-Silver | U.S. 45 Bk grade 
(2009) 
25. Jordan, Hmelo-Silver, Liu, & Gray (2013) U.S. 138 T*_g" orade 
26. Leach, Driver, Scott, & Wood-Robinson England ~200 Ages 5-16 
(1992) 
27. Leach, Driver, Scott, & Wood-Robinson England ~200 Ages 5-16 
(1995) 
28. Leach, Driver, Scott, & Wood-Robinson England ~200 Ages 5-16 
(1996a) 
29. Leach, Driver, Scott, & Wood-Robinson England ~200 Ages 5-16 
(1996b) 
30. Magntorn & Helldén (2007a) Sweden 23 Ages 10-11 
31. Magntorn & Helldén (2007b) Sweden 15 Ages 13-14 
32. Myers, Jr., Saunders, & Garrett (2003) U.S. 171 Ages 4-14 
33. Ozkan, Tekkaya, & Geban (2004) Turkey 58; 10 students 7 and 8 grade 
interviewed 
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34. Palmer (1997) Australia 123 Ages 12 and 16 

35. Papaevripidou, Constantinou, & Zacharia Cyprus 16 5" grade 
(2007) 

36. Simpson & Arnold (1982a) Scotland 578 Ages 11-16 

37. Simpson & Arnold (1982b) Scotland Varied based on task | Ages 11-16 

and age range 
38. Smith & Anderson (1984) U.S. 1 class 5a grade 
39. Tsoi (2011) China 140 Ages 8-10 


In attempting to code PCK from these articles, it became clear that the literature is heavily 
focused on aspects of student thinking, and within student thinking, primarily on misconceptions. 
Some studies also included prompts used to elicit student thinking; however, these studies 
typically concentrated on examining student thinking rather than the efficacy of particular 
instructional strategies. Nonetheless, these questions and tasks may offer guidance for first steps 
towards uncovering students’ initial ideas. Below, we summarize our findings organized by five 
fundamental ideas related to interdependent relationships in ecosystems. 


Producers make food for their growth using light, carbon dioxide from air, and 
water. 
Several studies examined student thinking related to the concept of producers. Study results 


often indicated that students are unaware of producers making their own food (Demetriou et al., 
2009; Leach et al., 1992). The notion that producers take in food, or nutrients, from their 
surroundings (e.g., the soil), as opposed to making food, is pervasive in student thinking 
(Adeniyi, 1985; Barman et al., 2006; Brody, 1993; Cetin, 2007; Helldén, 1998; Hogan & 
Fisherkeller, 1996; Leach et al., 1992, 1996a; Ozkan et al., 2004; Simpson & Arnold, 1982b; 
Smith & Anderson, 1984). For example: 


When considering what plants need in order to grow, students tended to portray plants as 
requiring what humans need in order to grow. For example, the ideas that plants "eat," 
"drink," and "breathe" occurred frequently. The students observe plants as being 
provided sunlight, water, and food externally, much like their idea of how they themselves 
take in food and water to grow, or stand in the sun to be warmer. (Barman et al., 2006, 
p. 75) 


Plants get energy from their food. This food is obtained from the soil, through the roots. 
(Simpson & Arnold, 1982b, p. 177) 


Other studies reported that some students do acknowledge that producers make food for their 
growth; however, they tend to hold an incomplete understanding of the process (Barman et al., 
2006; Helldén, 1998; Simpson & Arnold, 1982b; Smith & Anderson, 1984). These incomplete 
understandings often result from a combination of missing conceptions or misinformation, 
including students’ omission of one or more of the essential components (i.e., light, carbon 
dioxide, and water) needed to produce food. Incomplete understandings also arise from 
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confusion surrounding the source of these components. Helldén reported instances of such 
confusion in his longitudinal study: 


[The student] suggested for example at [age] 11 that the carbon dioxide came from the 
worms in the soil. (1998, p. 10) 


Smith and Anderson (1984) found that students thought of light as something to enhance plant 
health rather than a requirement for food production and survival. Interestingly, a classroom 
investigation designed to examine plants’ needs inadvertently introduced this misconception. 
The class carried out an investigation where they grew grass in the light and in a dark closet: 


When the students observed that grass began to grow in the dark, their view that plants 
need light to live and grow was shaken. After observing specimens of grass from the 
closet...the students nearly unanimously asserted that plants do not need light. The 
observation that the grass in the dark was actually taller than that in the light probably 
contributed to this belief. As the students continued to observe the plants...they used 
such words as “dark green,” and “stronger,” and “straight” to describe those in the 
light, while those in the dark they labelled “light green” or “yellow,” “ 
growing “in all directions.” This was consistent with the view that plants need light to 
be healthy, and more students appeared to develop this conception. (Smith & Anderson, 
1984, pp. 689-690) 


weaker,” and 


Additionally, students’ unfamiliarity with aquatic vegetation makes it difficult for them to fully 
understand the needs of producers in aquatic environments (Adeniyi, 1985; Brody, 1993, 1994; 
Brody & Koch, 1990): 


For most students, the apparent lack of vegetation in large bodies of water led to the 
misconception that plants do not live in water. (Adeniyi, 1985, p. 314) 


Matter and energy flow through ecosystems. Matter provides organisms the 
materials and energy necessary to function and grow. 
Several studies examined student thinking about trophic relationships and the flow of matter in 


ecosystems (Adeniyi, 1985; Gallegos, Jerezano, & Flores, 1994; Gotwals & Songer, 2010; 
Grotzer, 2009; Hogan, 2000; Hogan & Fisherkeller, 1996; Magntorn & Helldén, 2007; Ozkan et 
al., 2004; Tsoi, 2011). Tasks involving the construction and interpretation of food webs or a 
particular ecosystem were common approaches for exploring students’ ideas related to 
understanding and illustrating relationships. Illustrative examples of the prompts found to target 
the concept of relationships within ecosystems follow. 


Grotzer reported use of the following in student interviews: 
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Here is a picture of the forest. Here are some plants and animals from the forest. What 
are some ways that they are important to each other? (2009, p. 26) 


Demetriou et al. asked students to respond in writing to the following questions, based on their 
interpretation of a provided food web: 


a) What do snakes eat? 
b) Which species’ population will be affected if the sparrows disappear? 
c) What will happen if the corn disappears ? (2009, p. 183) 


Researchers found that some students conceptualize food web relationships at the individual 
level (1.e., a single predator and a single prey), as opposed to the population level (Bell-Basca et 
al., 2000; Leach et al., 1996b). For example: 


In this study many pupils, particularly between the ages of five and 11, were likely to talk 
about organisms in the singular, suggesting a relationship between one predator and one 
prey organism as opposed to relationships between populations. Pupils between five and 
seven did not show any evidence of conceptualizing groups of interdependent organisms 
in ecosystems; their only experience of organisms (and particularly mammals as 
represented in this task) was in the context of zoos, farms, storybooks and pets. (Leach et 
al., 1996b, 136) 


Additionally, young students often associated organisms’ eating habits with their size, strength, 
or perceived ferocity (Adeniyi, 1985; Gallegos et al., 1994; Grotzer, 2009). Related to this point, 
Gallegos et al. identified two commonly held student views: 


I. Animals are carnivorous if they are big and ferocious. 
2. Animals are herbivorous if they are passive or, frequently, smaller than the 
carnivorous animals. (1994, p. 268) 


Similarly, researchers found that students tended to believe that organisms eat all others below 
them in a food web (Grotzer, 2009; Hogan, 2000; Ozkan et al., 2004). A top-level consumer, for 
example, would be seen as having all other organisms in the ecosystem as a food source. 
Additionally, Grotzer characterized students as having “a tendency to focus on menu-driven as 
opposed to opportunity-driven feeding relationships” (2009, p. 16). That is, animals choose from 
among many options in selecting their prey. Similar trends appeared to exist in student ideas 
about organisms easily changing their diet based on food availability (Grotzer, 2009; Hogan, 
2002; Leach et al., 1992; Tsoi, 2011). 
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There were a number of instances, where it was clear that either experiences that 
students had or didn’t have, influenced how they reasoned about the concepts. If one does 
not have the opportunity to see how certain plants, seeds, and berries are seasonal in the 
woods and can be bought every day of the year in the local supermarket, the 
opportunistic nature of a food web is unlikely to come across without explicit attention to 
it. (Grotzer, 2009, p. 16) 


Some students appear to believe that organisms generally depend on humans to supply their 
needs (Demetriou et al., 2009; Leach et al., 1992, 1996b): 


Specifically, most [students] considered that ducks, fish and geese were human- 
dependent, declaring that they fed on bread that humans threw to them. (Demetriou et 
al., 2009, p. 185) 


Many pupils...thought that mice eat cheese that is given to them by people, and that 
rabbits eat carrots that are given to them, by people or taken from people’s vegetable 
gardens. (Leach et al., 1992, p. 97) 


The NGSS introduce the concept that organisms need food in grades K—2; however, in grades 3— 
5, the expectation deepens to students understanding that food provides animals with the 
materials and energy they need for body repair, growth, warmth, and motion. One study’s 
findings illustrate student thinking related to this progression: 


From the age of about eight, a growing number of pupils appreciated that food is an 
essential requirement for the growth of living things. However, the scientific explanation 
that the body matter of all organisms is “chemically transformed food” posed huge 
problems to learners. (Leach et al., 1996a, p. 31) 


Helldén found that students were inclined to view organisms as “the ‘end station’ for matter 
flowing through the ecosystem,” (1998, p. 8). Others also found this to be the case, particularly 
with the process of decay (Hogan, 2000; Leach et al., 1996a). Students appear to think in ways 
that run counter to the concept that decomposition eventually restores (recycles) some materials 
back to the environment. For example: 


The vast majority of pupils up to the age of 16 did not see any need to explain where all 
the matter goes during the process of decay (Leach et al., 1996a, p. 29) 


[The student] said that salt can get into freshwater, kill freshwater fish, snail and plants, 
then wash up onto land and get into soil where it harms grass and mustard plants, then 
goes into the crickets that eat the plants, and finally is excreted as animal waste. He 
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stopped short however, of tracing the excreted salt back into the ecosystem. (Hogan, 
2000, p. 26) 


Decomposition is the chemical breakdown of dead organisms (or organism parts) 
performed by some consumers, and is essential to a healthy ecosystem. 
Several studies found evidence that elementary-age students view decomposition as a process 


that occurs in the absence of decomposers (Cetin, 2007; Ero-Tolliver, Lucas, & Schauble, 2013; 
Grotzer, 2009; Helldén, 1998; Hogan & Fisherkeller, 1996; Leach et al., 1992, 1995). Within a 
subset of these studies, students conceptualized decomposition as either the physical 
disintegration (e.g., leaves being ground up into smaller parts, as opposed to chemical 
decomposition) or removal of a dead organism or its parts by humans (Ero-Tolliver, Lucas, & 
Schauble, 2013; Grotzer, 2009; Helldén, 1998; Hogan & Fisherkeller, 1996). Ero-Tolliver et al. 
suggested how students’ environments may influence their thinking:: 


Students said that the leaves “disappeared,” “died,” were “blown away,” or were taken 
away by trash collectors, all reasonable replies, given the experiences of these urban 
children. (2013, p. 2141) 


Other studies highlighted students’ uncertainty about what organisms perform decomposition in 
an ecosystem and particularly students’ tendency to not associate microbes with decomposition 
(Cetin, 2007; Grotzer, 2009; Leach et al., 1992). For example: 


On their pretests, students missed non-obvious, microbial causes of decay. Students 
talked about processes such as aging, weathering, [and] erosion as factors in the 
structural change of the things that decay. Or they referred to more obvious 
decomposers such as worms, animals or kids stepping on it, etc. (Grotzer, 2009, p. 14) 


Researchers suggest that limited exposure to decomposition, may contribute to students’ 
unfamiliarity with the phenomenon and the idea of decomposers as living organisms (Grotzer, 
2009; Leach et al., 1992): 


The pupils did not think that [mold] was alive...they thought that [mold] came from 
within decaying material as it decayed. (Leach et al., 1992, p. 174) 


Unless students have experiences witnessing decay, they do not necessarily have ways of 
realizing that decay does occur even if no children are around to step on the piles of 
leaves that they see on city sidewalks. The city sidewalks eventually get swept clean so 
students do not have the experience over time of witnessing the process. Time delays are 
difficult enough to deal with because we often stop attending to such processes, however, 
in this case, many of the students don’t have the experiences to support an understanding 
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of what happens over time. This complicates their opportunity to witness what happens 
and to realize that tiny microbes play a role. (Grotzer, 2009, p. 14) 


Ultimately, these alternate conceptions of decomposition may lead to students discounting the 
important function of decomposers in an ecosystem (Leach et al., 1992; Ozkan et al., 2004) and 
an incomplete understanding of the cycling of matter. 


Although the aforementioned studies found students to lack a complete understanding of 
decomposition and the role of decomposers, Leach mentioned leveraging a common student 
experience when trying to elicit student thinking about these ideas: 


“[The study] used decaying fruit as a stimulus because children in the trials of the probes 
seemed familiar with decay in this context. It may be that children would offer different 
explanations about the causation of decay in different contexts such as a decaying animal 
body.” (1992, p. 114) 


Others also drew on everyday experiences in order to ask students to make predictions or explain 
phenomena that they may not have considered otherwise. For example: 


What do you think will happen to the leaves on the ground in the autumn? (Helldén, 
1998, p. 6) 


What happens over time after a tree in the forest dies? (Grotzer, 2009, p. 6) 


John put four apples in a glass bowl, but then he forgot to eat them for two weeks. He 
noticed that they have a bad smell and have lost their original shape. Explain briefly 
why they can smell and lost their original shape. (Cetin, 2007, p. 7) 


Research indicated that pictures or time-lapse videos can also be useful stimuli for eliciting 
student ideas related to decomposition (Grotzer & Basca, 2003; Helldén, 1998; Leach et al., 
1995). One study utilized ongoing observation: 


First, the teacher put a single banana on each table where four students sat. The easy 
accessibility of the banana to sight (as well as smell and touch via occasional poking) 
encouraged students to notice gradual changes in its appearance from day to day, and 
the children drew and wrote descriptions of change in their notebooks. (Ero-Tolliver et 
al., 2013, p. 2143) 


Abiotic factors impact organisms’ ability to function and survive. 
Across several studies, students appeared to hold naive conceptions related to abiotic factors. 


These environmental conditions include, but are not limited to, light, temperature, moisture, 
amount of oxygen, nutrient availability, and salinity. Students often either do not recognize non- 
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living factors as part of an ecosystem (Brody, 1993, 1994; Magntorn & Helldén, 2007b; Ozkan et 
al., 2004), or commonly believe that organisms are not affected by these abiotic factors (Adeniyi, 
1985; Brody, 1994; Leach et al., 1996a). Hogan documented differences in how students 
understand the effects of abiotic factors. In some instances, students overlook indirect effects on 
organisms, such as the extended impacts of pollutants on an ecosystem; in others, students 
suggest that abiotic factors affect all organisms equally (Hogan, 2000). The tendency to 
overgeneralize also appeared in students’ understandings of particular abiotic factors. For 
example, Hogan (2000) found that students often viewed fertilizer as universally toxic. In 
contrast, Brody (1994) shared examples that demonstrate students’ inclination to disregard 
natural, biodegradable, or non-visible entities as pollutants. Brody also found that some 
students, although they recognized abiotic factors, viewed these factors as unchanging: 


[Students] were missing knowledge of how salinity and temperature may differ 
seasonally and in different parts of the marine environment. (1993, p. 13) 


The vast majority of the prompts found to address abiotic environmental conditions focused on 
producers’ survival and growth in varied conditions. As mentioned previously, prompts of this 
nature typically asked students to make predictions about what would happen to the plant 

(Helldén, 1998; Smith & Anderson, 1984), rather than focusing on the process of making food. 


All populations within an ecosystem are interdependent. 
Students’ uncertainty about the range of connections existing in food webs appeared in varied 


forms throughout the literature. Students’ thinking about food web disturbances appears similar 
to their conceptions of abiotic factor impacts. Several studies found that students think 
populations are affected only by those directly linked in a food web (Bell-Basca et al., 2000; 
Grotzer, 2009; Hogan, 2000, 2002; Hogan & Fisherkeller, 1996; Magntorn & Helldén, 2007b; 
Ozkan et al., 2004; Tsoi, 2011). One study noted: 


A main result was that 11 year old students tend to focus on local rather than extended 
effects within an ecological system. (Hogan, 2000, p. 27) 


Grotzer reported instances of this pattern in student thinking related to producers: 


In the pre-interviews and at the outset of the unit, students typically missed indirect 
effects—they didn’t realize that if all the green plants disappeared, it would affect just 
about everything in the food web, not only the things that directly eat green plants. 
(2009, p. 7) 


Additionally, Grotzer and Basca discussed the difficulty that elementary-age students face when 
trying to understand “underlying causal patterns”: 
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Non-obvious causes make it difficult for students to analyze interdependencies. Hidden 
agents may disguise a causal relationship or contribute to processes in unexpected ways. 
For example, the causal actions of microorganisms are not easily available to students 
without special tools and there is no particular reason they would assume that there is a 
causal mechanism they cannot see. When effects are removed in time and space from 
their causes, students have difficulty recognizing them as connected to the precipitating 
event (e.g., extended domino-like relationships in food web). (2003, p. 18) 


Similarly, students’ understandings also appear somewhat limited when examining the 
directionality of effects (Gotwals & Songer, 2010; Hogan, 2002; Leach et al., 1996b): 


Pupils are more likely to trace effects up through trophic levels than down through 
trophic levels. For example, pupils were likely to think that the removal of mountain 
lions would not affect the population sizes of other organisms, but that the removal of 
grass and crops would affect most other organisms. When primary consumers were 
manipulated, pupils were more likely to trace the effects up through the trophic levels to 
predators than down to producers. (Leach et al., 1996b, p. 137) 


Prompts focused on the interdependent nature of ecosystems most often posed hypothetical 
scenarios to elicit initial ideas, or referred to a previous investigation to assess understanding; in 
both cases, the provided context was typically the decline of a particular population or the 
introduction of pollutants. For example: 


The snakes in this place are all hit by an illness, so that they all die. The illness only 
affects snakes. What do you think might happen as a result of this? Explain as clearly as 
you can. (Leach et al., 1992, p. 93) 


For the pollutant effects task, students were given three cards labeled Fertilizer, Acid 
Rain, and Salt - the three pollutants they had used during their Eco-Column experiments. 
They were told to show all of the places that the pollutant would go within their web, 
what it would do there, and what would happen within the web as a result. (Hogan, 
2000, p. 24) 


Research also suggests that students differ in their grasp of how a decline in producers would 
have adverse effects on other organisms. Some researchers pointed to students’ missing 
conceptions about the role of producers and their importance to other organisms in an ecosystem 
(Brody, 1994; Brody & Koch, 1990; Grotzer, 2009; Hogan, 2002). In their discussion of 
statewide science assessment results, Brody and Koch (1990) noted the following responses to 
one multiple-choice item: 


Horizon Research, Inc. 16 May 2017 


...only 25% of 4th graders, 50% of 8th graders and 62% of 11 e graders knew that green 
plants are important to animals because they make food and give off oxygen. (p. 24) 


Others found that students viewed ecosystems as universally fragile, believing that a food web 
disturbance would affect all populations similarly (Grotzer, 2009; Ozkan et al., 2004): 


However, one of the issues that arose was extending the concept of connectedness too far. 
Once students got the idea of the connectedness in the food web, some of them applied it 
indiscriminately and thought that if any animal died, the whole web would collapse. 
(Grotzer, 2009, p. 13) 


If the size of one population in a food web is altered, all other populations in the web will 
be altered in the same way. (Ozkan et al., 2004, p. 98) 


CONCLUSION 


Our review suggests that the majority of the empirical literature about interdependence in 
ecosystems at the upper elementary level is focused on student thinking. Although a subset of 
studies focused on changes in student thinking as a result of a particular instructional approach or 
instructional unit, very few studies included enough detail for a teacher to implement the 
instruction studied. Therefore, the PCK we synthesized, like the literature itself, largely focused 
on student thinking. 


Patterns emerged across the aspects of student thinking discussed in the literature. In addition, 
we noticed even broader themes in student thinking that run across the targeted ideas. The first 
of these overarching themes is anthropomorphic thinking—that is, considering nonhuman 
entities to possess humanlike traits or motivations. An example includes thinking that producers 
take in food (like humans), rather than needing materials to make food. Anthropomorphism also 
emerges in student conceptions of trophic relationships, as students may apply their own 
experiences of selecting food at a grocery store or restaurant to the feeding behavior of 
organisms in an ecosystem. 


Upper elementary students also appear to view organisms as dependent on humans. The idea 
that organisms and processes rely on humans may result from students’ everyday experiences, 
particularly those in a garden, home, or zoo setting, which may reinforce this notion that 
organisms could not survive without human intervention. Similarly, students may attribute the 
disappearance of decaying parts of organisms to humans rather than decomposers. 


A third theme focuses on students’ incomplete understandings of the abstract—in the case of 
ecosystems, entities or processes that cannot be seen. For example, abiotic factors often go 
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unrecognized; Brody (1994) found that elementary-age students need to sense something 
physically in order to acknowledge its existence resulted in a limited understanding of pollutants. 
Similar patterns of unawareness and uncertainty exist in student thinking about the concepts of 
decomposition and producers making their own food. 


An additional, yet related, theme cutting across these concepts involves students’ tendency to 
focus on immediate interactions and effects. In feeding relationships, students often focus only 
on direct connections. When causes and effects are separated by time or space, students may 
find it difficult to pinpoint the connection. One example is that students tend to consider effects 
of abiotic resources only as they relate to the directly interacting population. Similarly, when 
students consider the flow of matter, few integrate the recycling of matter back to the 
environment and through the food web. Additionally, students tend to generalize their 
impressions of the familiar to all instances. For example, Hogan (2000) found some students to 
assume that all fertilizers are toxic based on their experiences. 


In summary, much of the student thinking discussed in this review is rooted in students’ 
everyday experiences, including both what they observe in the natural world and hear in 
conversation. The following section outlines associated implications, as well as broader 
considerations for teaching about interdependent relationships in ecosystems. 


IMPLICATIONS FOR INSTRUCTION 


As previously mentioned, students’ experiences observing and conversing about the natural 
world often shape the way they approach ideas about interdependence in ecosystems. For this 
reason, the use of precise language is particularly important when working to move students 
toward scientifically accurate understandings. For example, conveying producers’ needs by 
saying, “The plant wants light,” or “The plant is happy because it was watered,” both 
anthropomorphizes producers and treats these required materials for survival and growth as mere 
enhancements. Modeling evidence-based explanations and pressing for them can serve as a 
helpful practice in encouraging precise and scientifically accurate language use. 


The literature suggests that students need explicit opportunities to consider the effect of factors 
they cannot see or sense in other ways—for example, the effect of soil nutrients or pollutants on 
plant health. Such experiences could take the form of observing several model systems with 
varied conditions, or using an online simulation to manipulate an ecosystem and observe gradual 
changes in populations. In either case, instruction should clearly tie changes to underlying 
factors, with the goal of helping students see these factors as less abstract. 


Closely related, students likely need opportunities to consider the role of decomposers in 
recycling matter. Observing a plant in a sealed terrarium can be a powerful discrepant event for 
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students, as they may predict that the plant will die (Helldén, 1998). Seeing the plant survive 
creates a need for students to consider the sources of the materials that the plant is using to make 
food. Investigating these sources (e.g., decaying plant and animal matter) naturally leads to the 
question of how the materials are transformed into resources the plant can use to make food. 
Time-lapse videos of decay enable students to examine changes that would be more difficult to 
observe in a natural setting or a modeled system (Grotzer & Basca, 2003; Leach et al., 1992). 


Our review also has implications for broad concepts that might be emphasized in instruction. 
Each of the themes we identified encompasses student thinking that addresses multiple concepts 
related to interdependence. The NGSS envision three-dimensional learning that interweaves 
disciplinary core ideas, science practices, and crosscutting concepts. It is not surprising that the 
NGSS align the content unpacked in this review with the crosscutting concept of “Systems and 
System Models” (NGSS Lead States, 2013, p. 48). However, the themes presented in this review 
suggest that instruction focused on interdependent relationships in ecosystems could support 
students’ development of additional crosscutting concepts such as: (1) Cause and effect: 
Mechanism and explanation, and (2) Energy and matter: Flows, cycles, and conservation 
(National Research Council, 2012, p. 84). 
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Table A-1 


Article Summaries 


Author (Year) 


Where the 
study occurred 


N 


Age/Grade 
of subjects 


Summary 


1. Adeniyi (1985) 


Nigeria 


232 tested; subset of 26 
students interviewed 


Ages 13-15 


Used data collected from classroom 
observation, open-ended assessment 
responses, and clinical interviews to 
examine student misconceptions. 


2. Alao & Guthrie (1999) 


US. 


72 


5" grade 


Administered cognitive and motivational 
measures to examine predictors of 
conceptual understanding. 


3. Bailey & Watson (1998) 


England 


~100 


Ages 7-11 


Administered questionnaires to examine 
students’ attitudes toward drama/role 
play, as well as students’ understanding 
of ecological concepts. 


4. Barman, Stein, McNair, & Barman (2006) 


U.S. & Canada 


~2400 


K-8" grade 


Used data collected from teacher- 
conducted interviews to study student 
thinking about plants and plant growth. 


5. Bell-Basca, Grotzer, Donis, & Shaw (2000) 


US. 


60 


3" grade 


Interviewed students to examine the 
effects of an instructional unit and 
interventions on student thinking. 


6. Brody (1993) 


US. 


316 


4", 8", and 11" grade 


Conducted interviews in order to make 
comparisons of student understanding in 
various areas. 


7. Brody (1994) 


US. 


467 


4" 8" and 11! grade 


Conducted interviews to assess student 
knowledge of science and natural 
resource concepts in relation to ecological 
crises. 


8. Brody & Koch (1990) 


US. 


187 


4n git and 11" grade 


Conducted interviews to assess student 
knowledge of marine ecosystems. 


9. Cetin (2007) 


England & Turkey 


96 


Ages 14-15 (7" 
grade) 


Administered questionnaires to examine 
and make comparisons of student 
understanding of decomposition. 


10. Demetriou et al. (2009) 


Cyprus 


46 


Ages 9-10 (4" grade) 


Administered pre- and post-tests to 
examine the efficacy of a curriculum 
focused on trophic relationships. 
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11. 


Ero-Tolliver, Lucas, & Schauble (2013) 


US. 


23 


1* grade 


Administered pre- and post-instructional 
written assessment to examine students’ 
reasoning about the process of 
decomposition. 


12. 


Evagorou, Korfiatis, Nicolaou, & Constantinou 


(2009) 


Cyprus 


13 


5" and 6™ grade 
(Ages 11-12) 


Administered assessment tasks pre- and 
post-instruction to evaluate the impacts of 
using simulations on students’ 
development of seven discrete system 
thinking skills. 


13. 


Gallegos, Jerezano, & Flores (1994) 


Mexico 


506 


4"_6" orade 


Administered a three-task questionnaire 
to students to explore their thinking about 
trophic relationships. 


14. 


Gotwals & Songer (2010) 


US. 


318 


6" grade 


Collected data using written assessments 
and cognitive interviews to examine 
student reasoning related to food web 
disturbance. 


15. 


Grotzer (2009) 


US. 


99 


6" grade 


Conducted qualitative research in 
classrooms implemented a specified unit 
about causality in ecosystems. 


16. 


Grotzer & Basca (2003) 


US. 


60 


3" grade 


Conducted interviews to examine the 
impacts of an intervention on student 
understanding of ecosystem concepts. 


17. 


Helldén (1998) 


Sweden 


~25 


2>4_g" orade 


Conducted a longitudinal study; 
interviewed students to examine changes 
in their thinking. 


18. 


Hmelo-Silver, Eberbach, & Jordan (2014) 


US. 


311 


Middle school 


Administered assessments pre- and post- 
instruction to examine the effects of a 
technology-infused curriculum on 
students’ understanding of relationships 
in an aquarium ecosystem. 


19. 


Hogan (2000) 


US. 


52; 16 focal students 


6" grade 


Analyzed written tasks as well as 
interview tasks to examine student 
thinking before and after engagement in a 
unit centered on mini-ecosystems. 


20. 


Hogan (2002) 


US. 


24 


8" grade 


Conducted one-on-one interviews and 
administered a group task to explore 
students’ reasoning about an 
environmental management decision. 


Horizon Research, Inc. 


May 2017 


21. 


Hogan & Fisherkeller (1996) 


US. 


Curriculum piloted in 12 
classrooms; 8 students 
interviewed 


and 6" grade 


Conducted interviews in order to develop 
a coding scheme for analyzing student 
ideas. 


22. 


Jordan, Brooks, Hmelo-Silver, Eberbach, & Sinha 
(2014) 


US. 


66; data analyzed from 
35 students who 
completed all tasks 


7 


grade 


Analyzed student-drawn models and 
written responses to examine student 
understanding of ecosystem processes. 


23. 


Jordan, Gray, Brooks, Honwad, & Hmelo-Silver 
(2013) 


US. 


40 


7 


grade 


Coded and analyzed students’ drawn 
models of ecosystems to examine the 
effects of an intervention involving the 
use of an aquarium on student thinking. 


24. 


Jordan, Gray, Demeter, Lui, & Hmelo-Silver 
(2009) 


US. 


45 


7 


grade 


Analyzed student work samples focused 
on model aquaria to assess student 
understanding of ecological processes. 


25. 


(R. C. Jordan, Hmelo-Silver, et al., 2013) 


US. 


138 


7 


_8" orade 


Conducted classroom observations and 
administered assessments pre- and post- 
instruction to examine the efficacy of 
instruction focused on aquatic 
ecosystems. 


26. 


(Leach et al., 1992) 


England 


~200 


Ages 5-16 


Interviews, written tasks, and 
observations were used to examine 
student ideas about various ecosystem 
concepts. 


27. 


(Leach et al., 1995) 


England 


~200 


Ages 5-16 


Provides information related to the 
theoretical background, design, and 
methodology of the study. 


28. 


Leach, Driver, Scott, & Wood-Robinson (1996a) 


England 


~200 


Ages 5-16 


Provides findings from the study related 
to student ideas about the cycling of 
matter. 


29. 


Leach, Driver, Scott, & Wood-Robinson (1996b) 


England 


~200 


Ages 5-16 


Provides findings from the study related 
to student ideas about interdependence. 


30. 


Magntorn & Helldén (2007a) 


Sweden 


23 


Ages 10-11 


Conducted interviews to examine how 
student understanding of ecosystems 
developed over the course of instruction. 


31. 


Magntorn & Helldén (2007b) 


Sweden 


15 


Ages 13-14 


Conducted observations and interviews to 
examine students’ ability to transfer 
conceptual understanding across 
ecosystems following a multi-phase 
instructional unit. 
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32. 


Myers Jr., Saunders, & Garrett (2003) 


US. 


171 


Ages 4-14 


Conducted interviews to collect data 
about students’ understanding of animals’ 
needs. 


33. 


Ozkan, Tekkaya, & Geban (2004) 


Turkey 


58; 10 students 
interviewed 


7" and 8" grade 


Conducted interviews and administered 
assessments to examine the effects of 
using conceptual-change texts in 
instruction. 


34. 


Palmer (1997) 


Australia 


123 


Ages 12 and 16 


Conducted interviews to study the ways 
in which students apply the concept of 
interdependence. 


35. 


Papaevripidou, Constantinou, & Zacharia (2007) 


Cyprus 


16 


5" grade 


Used pre-test results to guide the 
development of instructional materials, 
and administered a post-test to look for 
changes in student understanding. 


36. 


Simpson & Amold (1982a) 


Scotland 


578 


Ages 11-16 


Conducted interviews and administered 
assessments to examine student thinking 
related to concepts considered to be 
requisite knowledge for learning about 
photosynthesis. 


37. 


Simpson & Amold (1982b) 


Scotland 


Varied based on task and 
age range 


Ages 11-16 


Conducted interviews and administered 
assessments to gather student ideas’ about 
photosynthesis and related biological 
processes. 


38. 


Smith & Anderson (1984) 


US. 


1 class 


5" grade 


Conducted a case study using data 
collected from observations of instruction 
and planning, teacher interviews, and 
assessments administered pre- and post- 
instruction. 


39. 


Tsoi (2011) 


China 


140 


Ages 8-10 


Administered questionnaires to gather 
data about students’ perceptions of sharks 
and related ecosystems ideas. 


¥ All connected to the same research project designed to study student thinking related to ecological concepts. 
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Table A-2 
PCK Extraction Codebook 


Code Description Extraction Rules Confidence Rating 
Misconceptions are student ideas that are in conflict with Extract all The confidence rating is about how confident we 
accepted scientific ideas. Misconceptions typically arise misconceptions from an are that this misconception is widespread among 
from students’ interaction with the physical world around _| article, even if identifying | 5" grade students based on the study in the article. 
them. A common misconception is that plants get their misconceptions was not If the point of a study was to identify student 
food from soil. Misconceptions are neither good nor bad, | the intent of the study. misconceptions and the article fares well in the 
but they do tend to be deeply ingrained in students’ Can modify or rapid SoE review, the misconception gets a high 
thinking. Some are part of a learning progression for a paraphrase article text for | confidence rating. All other misconceptions get a 
topic, suggesting that many students will have the clarity, brevity. For now, | low confidence rating. NOTE: when we 

Misconception misconception at some point as they develop full lump missing synthesize across studies, a misconception that 
understanding. conceptions with shows up several times with a low rating may 
misconceptions. Capture | receive a high rating based on the accumulation of 
related misconceptions evidence. 
separately when possible. 
When present, capture 
the cognitive source 
along with the 
misconception. 
In contrast to misconceptions, misinformation is an Extract all The confidence rating is about how confident we 
incorrect fact not derived from every day experience with | misinformation from an are that this misinformation is widespread among 
the physical world. For example, students might think that | article, even if identifying | 5" grade students based on the study in the article. 
an organism must occupy only one level (e.g., primary misinformation was not If the point of a study was to identify student 
consumer) in a food web. Students’ misinformation is the intent of the study. misinformation and the article fares well in the 
Misinformation probably not as deeply ingrained in their thinking as rapid SoE review, the misinformation gets a high 


misconceptions are. 


confidence rating. All other misinformation gets a 
low confidence rating. NOTE: when we 
synthesize across studies, misinformation that 
shows up several times with a low rating may 
receive a high rating based on the accumulation of 
evidence. 
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Code 


Description 


Extraction Rules 


Confidence Rating 


Teaching tips are pieces of advice for teachers and are 
bigger than an individual activity, things that can be useful 
for teachers to know when teaching the topic. For 
example, teachers may need to consider whether students 


Extract all idea-level 
considerations from an 
article, even if identifying 
tips was not the intent of 


If the point of a study was to identify teaching tips 
and the article fares well in the rapid SoE review, 
the tip gets a high confidence rating. All other tips 
get a low confidence rating. NOTE: when we 


See think of organisms in food webs as individuals rather than | the study. synthesize across studies, a tip that shows up 
representing populations. several times with a low rating may receive a high 
NOTE: If a tip can be associated with all big ideas in the rating based on the accumulation of evidence. 
topic, it should be coded as a unit-level consideration 
instead (see below). 
Unit-level considerations (ULCs) are broader than Extract all ULCs from an | If the point of a study was to identify ULCs and 
teaching tips and apply to the entire unit, but they should article, even if identifying | the article fares well in the rapid SoE review, the 
Unit-level not be broader than the unit (the latter might actually be ULCs was not the intent ULC gets a high confidence rating. All other 
. : pedagogical knowledge instead of PCK). For example, of the study. ULCs get a low confidence rating. NOTE: when 
Consideration ; ; ; : 
students tend to focus on local impacts and have difficulty we synthesize across studies, a ULC that shows up 
reasoning about an ecosystem on a larger scale. Code a several times with a low rating may receive a high 
ULC to all big ideas in the topic. rating based on the accumulation of evidence. 
Prompts are questions or tasks that teachers would pose to | Extract a prompt if the The confidence in a prompt is based entirely on 
their students in order to elicit their thinking in writing or | reviewer can envision a the content of the prompt (e.g., how well aligned it 
orally to use for formative purposes. teacher using it with is with the idea, how “usable” it is by a teacher), as 
(Some prompts that are appropriate for research purposes students as is. The “bar” | judged by the reviewer. For example, a prompt 
(used in interviews, etc.) may be inappropriate for for modifying prompts that contains wording that may be inaccessible for 
classroom use.) from article text is higher | students would receive a low confidence rating. 
Prompt than that for 


misconceptions. If 
prompts that accompany 
activities or activity seeds 
can stand independently, 
capture as prompts. If 
not, don’t. 
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Code 


Description 


Extraction Rules 


Confidence Rating 


Instructional activities are stand-alone, ready-to-use 
activities that teachers can use in their instruction “as is” 
with no additional written materials or training. Their 
purpose is to develop understanding of a big idea. That is, 


Extract an instructional 
activity if a teacher can 
use it as is—i.e., it has 
sufficient context and 


If the point of the article was to investigate the 
impact of an instructional activity, the confidence 
rating will be based on the findings of the article 
and a rapid SoE. If the instructional activity was 


eeuceoeel the article should provide enough information so that instructions. incidental, the confidence rating will be low. If 
Acar, teachers are able to implement the activity in their the article explicitly investigates the efficacy of an 
classrooms. The learning goal should be explicit or easily entire unit and uses an individual activity to 
inferred. illustrate the material, the activity would receive a 
NOTE: Eventually, we will categorize the activities into low rating. 
more general instructional strategies, such as lab 
experiments, simulations, readings. 
Not a ready-to-use instructional activity, but a fleshed out | Do not capture if seed is In order to have a high confidence rating, an 
idea for an activity. The seed must have enough unsuccessful in activity seed must meet all three of the following 
description to determine that it fits some big idea(s) and to | implementation or in criteria: 
AleriySeed give a reasonable expectation that teachers could develop | need of substantial ¢ Is it explained in a way that is clear and 
it into an activity. modifications in order to | accessible to teachers? 
be helpful ¢ Are students likely to learn targeted content 
from it? 
¢ Is it feasible? (time required, materials required) 
Summative assessment activities are stand-alone, ready- Extract an assessment ifa | 1. Ifthe assessment does not have reliability and 
to-use activities that teachers can use in their instruction to | teacher can use it as is-- validity info, it should receive a LOW rating. 
evaluate students. Le., it has sufficient (NOTE: if the reliability and validity info are 
context and instructions. in another article, the assessment should be 
Summative Briefly summarize the extracted from that article.) 
Assessment form and substance of the | 2. If the assessment does have reliability and 
Activity assessment, and if there's validity info, the rating should be based on 


a rubric, describe how it's 
structured, what kinds of 
factors it takes into 
account. 


that information. Reliability should be above 
0.7, and there should be at least one form of 
validity evidence. 
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Code 


Description 


Extraction Rules 


Confidence Rating 


Common Student 


Common student experiences are things that a teacher can 
capitalize on in instruction, knowing that there is a good 
chance that most students have similar experiences. For 
example, most students have observed decaying fruit. 
Common student experiences may be keyed to one big 
idea or more than one. 


Extract a common 
student experience if 
there is evidence in the 
article that most students 
come to instruction with 
the experience. An 


The confidence rating is based on a rapid SoE 
review. 


Experiences article that describes 
what just one student has 
experienced is not 
sufficient. Do not 
include previous 
instruction experiences. 
Developmental challenges are things that students struggle | Extract a developmental The confidence rating is based on a rapid SoE 
with that are broader than misconceptions. For example, challenge if there is review. 
5" grade students and younger may struggle to accept the evidence in the article 
existence of decomposers that are too small to see. that most students come 
Developmental challenges may be keyed to one big idea to instruction with the 
or more than one. challenge. 
Some developmental challenges may have associated 
Developmental ULCs. For example, we think that kids do not apply 
Challenge explanatory frameworks consistently, but rather that it is 


context specific (e.g., students may understand the particle 
model in the context of boiling water, but will not apply it 
to condensation on a cold drink can). The unit-level 
consideration is that teachers can't assume that just 
because kids use the particle model appropriately in one 
context, they will use it appropriately in another. 
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Code 


Description 


Extraction Rules 


Confidence Rating 


Learning 
Progression 


A learning progression will probably be identified 
explicitly in an article. A learning progression is at the 
topic level, so we do not need to code to individual big 
ideas. All misconceptions in a learning progression can be 
coded with the progression. 


Extract a learning 
progression if the article 
describes a sequence of 
increasingly sophisticated 
and scientifically 
accurate understandings 
and skills within a 
domain that learners 
develop over several 
years. 


The confidence rating is based on a rapid SoE 


review. 
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